The present disclosure relates to systems and methods for capturing, streaming, and/or playing back immersive content, such as to simulate an immersive three-dimensional (3D), virtual reality environment.
Head mounted devices for displaying immersive content can allow a user to turn his or her head and experience a corresponding change in the displayed scene. Such devices can allow users to adjust head position to view greater than about 180 degrees of a captured scene in at least one dimension, and can provide up to about 360 degree viewing in at least one dimension.
While some display devices allow for 360 degree viewing, at any given time human eyes are capable of perceiving a limited field of view corresponding to only a portion of the entire available viewing area. This field of view can be less than about 180 degrees in a horizontal dimension. Thus as the user turns their head, the playback device can adjust the displayed content to present the portion of the viewing scene corresponding to where the user is looking. For example, when the user is looking forward the display device presents a user with a sector of the scene centered about a forward direction, which can correspond to an angle of 0 degrees in a horizontal plane that is approximately parallel to the ground. When the user turns around to face backward, the display device presents the user with the sector of the scene centered about a backward direction, which can correspond to an angle of 180 degrees in the horizontal plane.
Immersive systems capable of generating 360 degree viewing scenes typically combine content captured using multiple cameras. The cameras can capture stereoscopic content including image content corresponding to left and right eye views to allow for a stereoscopic 3D viewing effect.
In order to provide a wide field of view, cameras in immersive content capture systems can be fitted with wide angle lenses. These can include fisheye lenses or other ultra wide angle lenses that do not produce a rectilinear image. Fisheye lenses can produce strong visual distortion including significant barrel distortion, and create a wide panoramic or hemispherical image, which can make them well-suited for use in immersive content systems.
Spatial distortion, chromatic distortion, and/or other optical properties may vary from lens to lens and/or from camera to camera due factors including lens imperfections and differences between positioning of the lens relative to the sensor in the camera. This can make it difficult to reliably use images captured by cameras with fisheye lenses and to seam images together that are captured by adjacent cameras/lenses. In the case of stereoscopic images, for example, where separate left and right eye images are captured and then presented during playback to produce a 3D effect to the viewer, distortions and differences between the cameras used to capture the left and right eye images may be problematic and/or degrade the stereoscopic image quality if left unaddressed.